Do tests measure what they are supposed to? On validity of listening comprehension tests

Trafność jest jednym z najważniejszych kryteriów poprawności testu. Jak stwierdza Bolesław Niemierko, trafność gwarantuje, że „zmierzono wszystko to i tylko to, co należało zmierzyć” (1999: 178), a więc test nie sprawdza ani za dużo, ani za mało. Celem artykułu jest przybliżenie pojęcia trafności testu językowego zarówno z teoretycznej, jak i praktycznej perspektywy. Artykuł ma za zadanie odpowiedzieć na następujące pytania: Na czym polega trafność testu? Jak można rozpoznać, czy dany test sprawdzający rozumienie ze słuchu jest trafny? Jak wygląda realizacja kryterium trafności w praktyce, na przykładzie testowania umiejętności rozumienia ze słuchu na egzaminie maturalnym z języka niemieckiego?Validity is considered the most important criterion for the quality of a language test. As Bolesław Niemierko (1999: 179) states, validity guarantees that “everything and only what was supposed to be measured was measured”. The aim of this article is to present the validity notion from the theoretical as well as from the practical point of view. The paper should answer the following questions: What does test validity mean in practice? How can it be recognized that a listening comprehension test is valid? Do school leaving exams in German fulfill the validity criteria

Was meinst du damit?. Ein paar Bemerkungen über die Bedeutung und Entwicklung pragmatischer Kompetenz in der Fremdsprache

The proper identification of a function/goal of an utterance in a particular interaction (speech act) plays an important role in comprehension. The main aim of the article is therefore to investigate if and to what extent the reading and listening tasks offered in some secondary school English textbooks develop the skill to understand communicative intentions. The paper also gives a brief insight into the speech-act theory and the content of some important curricular documents, ending up with a few suggestions for classroom exercises.Die richtige Identifikation des Ziels einer Aussage in der kommunikativen Interaktion (Sprechakt) spielt in der fremdsprachigen Kommunikation eine bedeutende Rolle. Ziel des Beitrags ist es zu untersuchen, ob und inwieweit Lese- und Höraufgaben, die in modernen Englisch-Lehrwerken für die Oberstufe angeboten werden, entwickeln diese Fähigkeit. Der Artikel gibt also einen Einblick in die Sprechakttheorie und den Inhalt curricularer Dokumente. Abschließend werden auch ein paar Unterrichtsaufgaben zur Entwicklung der pragmatischen Kompetenz präsentiert

Testowanie sprawności receptywnych w standaryzowanych testach języka angielskiego i niemieckiego : próba analizy porównawczej

According to the psycholinguistic theory, reading and listening comprehension are active processes of seeking and creating a text meaning which take part in a reader’s and a listener’s cognitive systems, and involve different types of knowledge. The aim of this doctoral dissertation is to examine if and to what extent this theoretically and empirically grounded definition is reflected in teaching and testing of receptive skills. Furthermore, the thesis presents the issue of measuring comprehension in standardized language tests. Chapter One opens with a review of past and current psycholinguistic theories of comprehension. Reading and listening are presented as multilevel text processes which involve inferring and anticipating, and which are determined by schemas, memory and attention. The project also presents the differences between the auditory and the written texts comprehension, both in a native and a foreign language. The chapter ends with a list of practical implications for testing receptive skills. Chapter Two aims to indicate some important issues with regards to evaluation of reading and listening comprehension skills. It begins with the presentation of qualities of language tests, followed by the discussion of possibilities and limitations of testing comprehension abilities. Next, various testing factors are presented which may affect the comprehension process and its outcome in a testing situation. Chapter Third investigates the aims of developing and assessing receptive skills enumerated in a variety of important Polish and German language documents such as; Common European Framework of Reference for Languages, secondary school curricula, testing standards and teaching programs. Based on the fourth-level concept of comprehension which includes; 1) decoding, 2) understanding of explicitly stated information, 3) understanding of implicitly stated information, 4) critical understanding – evaluation, the analysis shows significant qualitative and quantitative differences between them and considerable lacks in their content. A great number of analyzed aims are not based on any psycholinguistics theory and do not provide any information about the achieved level (degree) of comprehension. The following major part of the dissertation (Chapter Four and Five) explores testing constructs and techniques on the basis of a broad spectrum of reading and listening tests. Chapter Four presents tests developed within such research projects and programmes as PISA, PIRLS, DESI, French and English as the first foreign language at the lower secondary level, run by Institute for Educational Progress in Berlin. In the analyzed reading and listening constructs, comprehension is perceived as a complex process of graded depth. Native-language reading tests (PISA, PIRLS, and DESI) assess four-level comprehension consisting of interpretation and evaluation of the text, whilst testing receptive skills in a foreign language is reduced to three levels of comprehension. The analysis of certificate tests in English and German on the B1-B2 level in Chapter Five reveals that tests in English measure a broader range of skills using more varied testing methods than equivalent tests in German. The third level of comprehension, i.e. interpretation is tested only to a certain degree, depending on the test (English/German) and its level (B1/B2), whereas evaluation is not tested at all. Chapter Six is devoted to a more detailed analysis of Polish school leaving reading and listening tests in English and German (2011), followed by the analysis of school leaving tests in English from the Thuringia and the Bavaria regions in Chapter Seven. It includes texts, test items and test methods, as well as recorded speech. The most important conclusions drawn from the analysis presented in the last two chapters are as follows: – Polish school leaving tests measuring receptive skills in English and German lack in construct and curricular validity. – There are significant qualitative differences between Polish school leaving tests in English and in German. Many of the examined reading and listening test items in German are wrongly constructed and some texts have been unskillfully adapted. – In Polish tests, close tasks are used exclusively, which in many cases demand only localization and recognition of the information, whereas the analyzed German tests include a variety of different tasks. Applied open techniques of testing allow a learner to present their own product of comprehension, such as text interpretation and their personal opinions. – Polish school leaving tests focus on the lowest levels of comprehension: decoding and understanding of explicitly stated information. None of the interpretation skills mentioned in the Polish examinations standards is tested. The reading tests from Thuringia and Bavaria measure full comprehension involving interpretation and evaluation of the text. Listening tests, however, evaluate only two (Thuringia) or three levels of comprehension (Bavaria). The recordings used at the Polish school leaving examination present unnatural sounding speech resembling written rather than spoken language, whereas in Thuringia and Bavaria learners listen to more realistic, spoken language. The requirements of Polish school leaving tests are evidently decreased in relation to the real needs and abilities of the learners. The dissertation ends with a list of conclusions, proposals and demands which could improve validity und usefulness of the Polish school leaving tests in reading and listening, and whose implementation seems to be necessary in the near future

Journalistische Texte in der Wirtschaftskommunikation: Welche Texte sind populärwissenschaftlich?

This article aims at presenting journalistic texts within the economic communication and answering the question: What kind of journalistic texts appearing in the economic communication should be considered popular scientific. Because of their comprehensible style and non-specialist audience, newspaper articles concerning economic issues are usually regarded as popular scientific texts. This is however a far-reaching simplification as newspaper articles differ considerably in their linguistic form from short news items about current economic affairs to long reports on important matters